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Abstract 

Traditional NoSQL systems scale by sharding data 
across multiple servers and by performing each opera¬ 
tion on a small number of servers. Because transactions 
on multiple keys necessarily require coordination across 
multiple servers, NoSQL systems often explicitly avoid 
making transactional guarantees in order to avoid such 
coordination. Past work on transactional systems control 
this coordination by either increasing the granularity at 
which transactions are ordered, sacrificing serializability, 
or by making clock synchronicity assumptions. 

This paper presents a novel protocol for providing 
serializable transactions on top of a sharded data store. 
Called acyclic transactions, this protocol allows multiple 
transactions to prepare and commit simultaneously, im¬ 
proving concurrency in the system, while ensuring that 
no cycles form between concurrently-committing trans¬ 
actions. We have fully implemented acyclic transactions 
in a document store called Warp. Experiments show 
that Warp achieves 4x higher throughput than Sinfo- 
nia’s mini-transactions on the standard TPC-C bench¬ 
mark with no aborts. Further, the system achieves 75% of 
the throughput of the non-transactional key-value store it 
builds upon. 

1 Introduction 

NoSQL systems have become the de facto back-end for 
modern applications because they allow unprecedented 
performance at large scale. The defining characteristic 
of these systems is their distributed architecture, where 
the system shards data across multiple servers to improve 
scalability. To further improve scalability, these sys¬ 
tems typically avoid cross-server communication, which 
makes it difficult to implement ACID transactions. 

Yet, distributed transactions require coordination 
among multiple servers. In traditional RDBMSs, trans¬ 
action managers coordinate clients with servers, and en¬ 
sure that all participants in multi-phase commit protocols 
run in lock-step. Such transaction managers constitute 


bottlenecks, and modern NoSQL systems have eschewed 
them for more concurrent implementations. Scatter 1211] 
and Google’s Megastore |@] shard the data across dif¬ 
ferent Paxos groups based on their key, thereby gain¬ 
ing scalability, but incur higher coordination costs for 
actions that span multiple groups, and require that op¬ 
erations on the same group be sequenced by the same 
Paxos instance. Google’s Spanner m combines tradi¬ 
tional locking techniques with a novel TrueTime API to 
provide fast read-write transactions, and lock-free reads. 
Many recent systems propose moving pieces of the trans¬ 
actional programs themselves to the server. Calvin ifs^ 
serializes all transaction inputs via a consensus proto¬ 
col, and then deterministically executes application code 
on servers. Lynx 0 and Rococo S break down the 
transaction into multiple atomic fragments, and employ 
static analysis to detect opportunities for reordering the 
shipped code components. Both techniques rely upon a 
priori knowledge and analysis of the transactions. 


This paper introduces Warp, a NoSQL system that en¬ 
ables efficient multi-key transactions with ACID seman¬ 
tics. Warp offers, to our knowledge, the highest degree 
of concurrency in a general purpose serializable NoSQL 
data store. Further, it achieves throughput far in excess 
of previous systems, approaching 75% of the throughput 
of the system on which it builds. The key insight that en¬ 
ables these performance improvements is a commit pro¬ 
tocol called acyclic transactions, which allows the sys¬ 
tem to order transactions on-the-fly without any prior 
static analysis, and without moving code to the server. 

Three techniques, working in concert, enable acyclic 
transactions to achieve its high scalability and perfor¬ 
mance. First, acyclic transactions reduce coordination 
costs by reducing the number of transactions that are or¬ 
dered to the minimum necessary to enforce serializabil¬ 
ity. Transactions that operate on disjoint data or whose 
executions do not overlap in time will incur zero coor¬ 
dination costs. Warp orders only those transactions that 
concurrently operate on overlapping data, and does so 
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with minimal overhead. 

Second, Acyclic transactions achieve scalability by 
arranging servers into data-dependent, dynamically- 
determined chains, where each chain contains, solely, 
those servers which store data affected by the transac¬ 
tion. This structure, avoids bottlenecks at transaction 
managers and improves performance by cutting commu¬ 
nication costs. 

Finally, acyclic transactions improve performance by 
allowing multiple overlapping transactions to proceed in 
parallel under the right conditions. Locking protocols in¬ 
herently limits concurrency, while traditional optimistic 
two-phase protocols can only prepare one transaction per 
key at a time, because all reads must be validated in the 
first phase before the second phase may begin. In con¬ 
trast, Warp enables multiple transactions to prepare si¬ 
multaneously by taking advantage of the inherent serial¬ 
ization in its data-dependent chains. 

Overall, this paper makes three contributions. First, 
we outline a novel protocol for providing efficient, one- 
copy serializable transactions on a distributed, sharded 
data store. Our protocol provides an unprecedented level 
of concurrency and scalability without any synchronic- 
ity assumptions or static analysis. Second, we describe 
our implementation of the commercially available Warp 
key-value store, including the design of the client. The 
system has been fully implemented and provides lan¬ 
guage bindings for C, C-H-, Python, Java, Ruby, Go, 
and Node.JS. Third, we show through macro- and micro¬ 
benchmarks that Warp can provide higher throughput 
than alternative designs, with fewer aborted transac¬ 
tions. Specifically, Warp achieves a throughput that is 4 x 
higher, with 5 x lower latency, than mini-transactions |01, 
the closest existing approach, on the TPC-C benchmark. 
The system achieves 75% the throughput of the underly¬ 
ing non-transactional key-value store upon which Warp 
builds. 

The rest of this paper is organized as follows. Sec¬ 
tions |2] and [3] describe the acyclic transactions protocol 
and our implementation of Warp. Section|4]evaluates the 
performance of Warp. Section|5]surveys existing systems 
and related work and Section|6]concludes. 

2 Design 

Warp builds upon the HyperDex key-value store by 
modifying the existing components to provide transac¬ 
tional guarantees. Warp’s architecture is based on Hy¬ 
perDex. To that end. Warp consists of three components: 
the coordinator, clients, and storage servers. The coor¬ 
dinator maintains the meta-state for the system, specif¬ 
ically, the partitioning of the key space across storage 
servers. Clients issue requests directly to the storage 
servers, where a request may affect a single object, or 
span multiple objects. Each storage server maintains a 



Figure 1: Warp’s architecture consists of storage servers, the 
coordinator, and the client library. The coordinator maintains 
the partitioning of the key space across servers, and supplies 
this mapping to the client library. The client library uses this 
mapping to directly contact storage servers. 


subset of keys in the system; collectively, the storage 
servers hold all data stored by the system. Figure [T] il¬ 
lustrates Warp’s overall architecture. 

The Warp client library provides isolation by opti¬ 
mistically performing read and write operations against 
local state, and verifying that this state remains the same 
at commit time. To perform a read, the library retrieves 
the requested data from the storage servers and caches 
the object within the local transaction’s state, called the 
transaction context. Subsequent reads within the trans¬ 
action will be satisfied by the transaction context, if pos¬ 
sible. Write operations executed within the transaction 
are not visible on the servers immediately. Instead, they 
are saved to the transaction context to be written at com¬ 
mit time. Multiple writes to the same key will overwrite 
the locally saved object. Storage servers are unaware of 
any modifications written within a transaction until the 
client commits the transaction. To commit the transac¬ 
tion, the library submits the set of all objects read and 
all objects written to the storage servers using the acyclic 
transactions commit protocol. 

2.1 Commit Protocol 

The acyclic transactions commit protocol processes 
clients’ transactions, and ensures that they either commit 
in an atomic, serializable fashion, or abort with no effect. 
The protocol does this for each transaction by simulta¬ 
neously validating the values optimistically read by the 
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client library, and ordering the transaction with respect 
to other concurrently executing transactions. While it is 
relatively easy to validate a transaction by comparing the 
values observed by the client to the latest values in the 
data store, it is considerably more difficult to order trans¬ 
actions across multiple servers. The former is a local 
check that each server may independently perform dur¬ 
ing transaction processing, while the latter requires that 
multiple servers coordinate to agree upon the serializable 
order of transactions. 

The key insight of the acyclic transactions protocol is 
to arrange the servers for a transaction into a chain, and 
to validate and order transactions using a dynamically- 
determined number of passes through this chain. Com¬ 
pared to traditional commit protocols which use fan- 
out/fan-in communication patterns, acyclic transactions 
pass messages forward or backward between adjacent 
nodes in the chain. This ensures that there is at most 
one server actively processing each transaction at any 
one time. By limiting the parallelism present in a single 
transaction, acyclic transactions enable each server to lo¬ 
cally make a binding decision about the fate of the trans¬ 
action they are processing, and propagate that decision to 
the next server in the chain. Globally, this enables multi¬ 
ple transactions which modify the same data to execute in 
parallel—transactions whose execution other techniques 
would serialize—because each pair of concurrent trans¬ 
actions is ordered by exactly one server that can decide 
their order without communicating with other servers. 
Any decision made by a server will be carried to, and 
enforced by, the remaining servers in the chain. 

The protocol consists of multiple distinct processes 
that work in concert to commit transactions. To commit 
the transaction, the client library determines the chain of 
servers which process the transaction’s commit. These 
are only those servers that house the data involved in 
the transaction. The servers in this chain follow simple 
rules to commit the transactions; a server passes a trans¬ 
action forward in the chain only when the server may 
commit that transaction; otherwise, the server sends ei¬ 
ther an abort or a retry request backward in the chain. 
Transactions will be aborted when they fail to validate 
the clients’ reads, and will be retried to ensure the order 
between concurrent transactions is serializable. When 
the transaction passes completely through the chain in 
the forward direction, the last server in the chain final¬ 
izes the transaction by sending a commit message back¬ 
wards through the chain. This commit message instructs 
servers to persist the transactions to disk, and to clean up 
any transient state related to the transaction. 

2.1.1 Chain Construction 

Clients use the transaction’s context to construct a chain 
to commit the transaction. To ensure that servers process 
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Figure 2: Clients deterministically construct dynamic chains 
based upon the keys read and written by transactions. In this 
example, a client submits 7j, Ti, and Tj. Transactions 7j and 
72 operate on disjoint keys, {k/^,kfj} and {kp,kj} respectively. 
Tj touches all four keys and forms a chain that includes the 
chains of T’l and T 2 


transactions’ operations in a predictable order, the client 
library sorts the keys read or written by a transaction in 
lexicographical order, and maps this sorted list onto a 
set of storage servers. Because sorting is a deterministic 
process, transactions with multiple keys in common pass 
through their shared set of servers in the same order. 

Figure |2] shows how transactions that read and write 
the same keys have overlapping chains. Transaction T’l 
reads key kn and writes key kA, while transaction T 2 
reads keys kp and kj- Transaction Tt, writes keys kA, kn, 
kp, and kp. The object-to-server mapping dictates that 
Ti’s chain pass through servers so and S 2 because these 
servers hold objects kA and kn respectively. Similarly, Tp 
forms a chain through S 5 and sg. Tj writes the same keys 
touched by T’l and Tp and has a chain that passes through 
the same servers as transactions T’l, and Tp. 

Constructing chains in this way makes it straightfor¬ 
ward to order transactions that concurrently operate on 
the same data. The first server in common between two 
transactions’ chains can order any two overlapping trans¬ 
actions, and notify all subsequent servers in both chains. 
The mapping maintained by the coordinator ensures that 
transactions with data in common will pass through a 
common set of same storage servers, even when the map¬ 
ping is updated to reflect system membership changes. 
Inversely, when two chains do not overlap, there is no 
need to directly order their transactions, because they 
necessarily operate on disjoint data. 
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2.1.2 Validation 


The validation step ensures that values previously read 
by the client remain unchanged until the transaction com¬ 
mits. To do this, servers ensure that the value the client 
read during its transaction is the same value currently 
in the data store, and that no concurrent transactions 
change the value that the client read. Specifically, servers 
check each transaction to ensure that it does not read val¬ 
ues written by, or write values read by, previously vali¬ 
dated transactions. Servers also check each value against 
the latest value in their local store to ensure that the 
value was not changed by a previously-committed trans¬ 
action. Thus, acyclic transactions employ optimistic con¬ 
currency control 02611491] . 

Servers perform validation for each transaction before 
forwarding it to subsequent servers in the chain. This 
ensures that at any step in a transactions’ processing, 
a prefix of the chain guarantees that the transaction is 
valid and will remain valid until the transaction commits 
or aborts. Storage servers will abort subsequent trans¬ 
actions whose writes would invalidate previously valid 
transactions. Consequently, when a transaction reaches 
the last server in the chain, that server can decide to com¬ 
mit or abort the transaction without consulting any other 
server—the protocol guarantees to the server that every 
previous server will be able to commit the transaction. 

When a server determines that a transaction does not 
validate, the server aborts the transaction by sending an 
abort message backwards through the chain. Each server 
in the prefix aborts the transaction and forwards the abort 
message until the message reaches the client. These 
servers remove the transaction from their local state, en¬ 
abling other transactions to validate in its place. 


2.1.3 Ordering 

The central task of the acyclic transactions protocol 
is to establish a serializable order across all validated 
transactions. While the protocol could simply serial¬ 
ize all transactions—which would maximize spurious 
coordination—it instead relies upon the observation that 
a set of transactions are serializable if the dependency 
graph of their relative orders is free of cycles. Each edge 
in this graph specifies a pair of transactions and the rel¬ 
ative order between them. We refer to the transactions 
at each endpoint of an edge as a conflicting pair, be¬ 
cause one transaction contains a write operation on a key 
which was read or written by the other. Consequently, 
every conflicting pair has at least one, and possibly sev¬ 
eral, keys that are common to both transactions. 

The difficulty in ordering these conflicting pairs lies 
not in resolving pairwise relationships, but in ensur¬ 
ing that every pairwise ordering is consistent with the 
globally serializable order. Resolving the order across 
multiple pairs is a considerably more complex task, be¬ 
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Eigure 3: Ordering transactions in a serializable fashion across 
multiple servers is difficult because of the possibility of dis¬ 
tributed cycles. In this figure, the three transaction’s chains 
overlap on servers .si, ^ 2 . but that no one server handles all 
three transactions. The acyclic transactions protocol prevents 
these transactions from forming the cycle shown on the right. 


cause interactions between transactions can span multi¬ 
ple servers. In these cases, it is possible that no sin¬ 
gle server would have have the requisite view to detect 
and prevent a cycle in the graph. Eor example, imag¬ 
ine three transactions across keys k^, ks, and kc, where 
each transaction writes to a different pair of the keys: 
{kA,kB), {kB,kc), and {kc,kA). If the system only ordered 
the transactions pairwise, the three transactions could be 
committed in a non-serializable order, because no single 
key is written by all three transactions. Eigure |3] depicts 
this example and highlights the problematic execution 
that results in a cycle between the transactions. 

Servers ensure that all transactions commit in a se¬ 
rializable order across servers by embedding ordering 
information, called mediator tokens, into transactions. 
Mediator tokens are integer values that are assigned to 
transactions by the heads of chains. Because media¬ 
tor tokens are integers, servers may determine the rela¬ 
tive order of conflicting pair by comparing their media¬ 
tor tokens. A simple invariant that ensures serializabil- 
ity is to commit conflicting pairs in the order specified 
by their mediator tokens. Eor example, if the mediator 
tokens for the conflicting pair (7x, 7y) have the relation¬ 
ship mediator(73f) < mediator(7V), then all servers order 
the transactions such that Tx commits before TV. 

The acyclic transactions protocol relies on this invari¬ 
ant to order transactions. Upon receipt of a transaction 
passing forward through the chain, a server compares the 
transaction’s mediator token to the largest mediator token 
across all transactions that previously read or wrote any 
of the current transaction’s objects. If the current media¬ 
tor token is larger than the previous token, the transaction 
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is forwarded to the next server in the chain. If, how¬ 
ever, the mediator token is less than the previous token, 
a “retry” message is sent backwards in the chain to the 
head, where the transaction will be retried with a larger 
mediator token. 

2.1.4 Generating Mediator Tokens 

At first glance, mediator tokens may resemble times¬ 
tamps that are typically used by transaction commit pro¬ 
tocols, but mediator tokens are signihcantly more flex¬ 
ible. Systems base d up on timestamps, whether they be 
logical timestamps 1291] or synchronized wall clocks, im¬ 
pose restrictions on how timestamps may be generated. 
Namely, timestamps must be generated in a monotonic 
fashion so that the system never moves backward in 
time. Failing to preserve the monotonicity of timestamps 
would permit transactions to commit in an unserializable 
fashion. Mediator tokens impose no such restrictions on 
token generation. 

The flexibility of mediator tokens permits a wide ar¬ 
ray of implementation strategies. For example, a sim¬ 
ple token generation strategy would be to always set a 
transaction’s initial token to zero, choosing successively 
larger values on each subsequent pass. Another strategy 
would be to generate tokens at random and ensure that 
each subsequent pass draws from a range of tokens that 
are strictly greater than the token of the previous pass. 
A strategy that limits the number of retries is for each 
server to maintain a counter to generate mediator tokens. 
Servers may generate a new mediator token by reading 
the counter’s current value and incrementing the counter. 
When a server sees a mediator token that is greater than 
the next value to be generated by its counter, it may ad¬ 
vance the counter to be greater than this other token. Be¬ 
cause mediator tokens are flexible, storage servers do not 
need to carefully manage or preserve this counter during 
server failover, and do not need to synchronize counters 
across servers. 

2.2 Fault Tolerance and Durability 

In a large-scale deployment, failures are inevitable. 
Acyclic transactions accommodate a natural way to over¬ 
come such failures. Specifically, acyclic transactions per¬ 
mit a subchain of / -f 1 replicas to be inlined into the 
longer chain in place of a single data server. This allows 
the system to remain available despite up to / failures 
within a subchain. Chain replication maintains a well- 
ordered series of updates within each subchain. Opera¬ 
tions that traverse the acyclic transaction chain in the for¬ 
ward direction pass forward through all inlined chains. 
Likewise, operations that traverse the chain in reverse 
traverse inlined chains in reverse. 

The notion of fault-tolerance provided by acyclic 
transactions is different from the notion of durability 


within traditional databases. While durability ensures 
that data may be re-read from disk after a failure, the sys¬ 
tem remains unavailable during the failure and recovery 
period; in contrast, acyclic transactions’ fault tolerance 
mechanism ensures that the system remains available so 
long as the number of failures remains below the config¬ 
ured threshold. 

2.3 Atomicity, Consistency, Isolation 

The protocol guarantees atomicity, consistency, and iso¬ 
lation for all transactions. These properties naturally 
follow from the one-copy serializability upheld by the 
protocol. Each transaction completes in its entirety at 
a well-defined point in the partial order, where its ef¬ 
fects are either completely visible to subsequent trans¬ 
actions, or it aborts without effect. Every server ensures 
that the stored objects are well-formed and match their 
data types. Overall, acyclic transactions guarantee that 
operations within a transaction execute with mutual ex¬ 
clusion from each other, as if there were a single giant 
lock protecting the database. 

2.4 Correctness 

By leveraging a fault tolerant system coordinator, acyclic 
transactions uphold both liveness and safety in the pres¬ 
ence of up to / faults. Specifically, acyclic transactions 
maintain serializability at all times, and will eventually 
commit or abort every transaction assuming at most / of 
the / + 1 replicas of the data remain non-faulty. In this 
section we demonstrate how the acyclic transactions pro¬ 
tocol maintains these safety and liveness properties. 

Safety; The acyclic transactions protocol provides se¬ 
rializability by ensuring that the final system state is 
equivalent to a serial schedule. The protocol upholds this 
guarantee by ensuring that the dependency graph across 
all transactions is acyclic. Intuitively, every conflicting 
pair directly corresponds to an edge in this graph, while 
the mediator tokens enforce the anti-cycle property. 

Because non-conflicting pairs operate on disjoint sets 
of data, the conflicting pairs are the only transaction pairs 
whose order must be carefully managed. A conflicting 
pair of transactions is committed in the same order on 
all common servers in order to ensure that operations to 
their shared state are applied in the same order. Servers 
use mediator tokens to decide the commit order for trans¬ 
actions in a conflicting pair; every server will enforce the 
order specified by the transactions’ tokens. 

Globally, mediator tokens preserve transitive relation¬ 
ships across transactions, ensuring that cycles cannot 
arise in the dependency graph. The dependency graph 
is structured such that each edge is a conflicting pair 
{Tx,Ty) such that either mediator(7j:) < mediator(7j,) or 
mediator(7j:) > mediator(7j,). In the former case, the 
graph will contain an edge Tx Tj,, while in the latter 
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case, the graph will contain edge Ty Tx- Any directed 
path will consist of edges such that the mediator token 
for the source is less than the mediator token for the des¬ 
tination. Transitively, the start of the path must have a 
mediator token less than the end of the path. Thus, it 
is impossible for the graph to contain a cycle, because a 
cycle would imply that there exists a directed path—and 
thus, a directed edge—from a transaction with a higher 
mediator token to a transaction with a lower mediator 
token. Because the system prohibits committing trans¬ 
actions in an order that contradicts the ordering estab¬ 
lished by their mediator tokens, it is impossible for such 
an edge, and thus a cycle, to exist. 

Liveness: The protocol remains available for process¬ 
ing transactions during a bounded number of server fail¬ 
ures. Specifically, the protocol will always be able to 
commit or abort a transaction so long as at most / servers 
fail of the f +l servers assigned to replicate each key. 
To enable the system to detect and correct for these fail¬ 
ures, acyclic transactions make use of a fault tolerant 
coordinator, which may be built using standard tech¬ 
niques This coordinator acts as a shepherd 

for the system, guiding it toward a stable state, even as 
servers fail or become otherwise unavailable. 

The system overcomes failures by removing failed 
servers from the chains for actively propagating trans¬ 
actions. For each failure, the coordinator issues a new 
configuration that lists the server as failed. Non-faulty 
servers may consult this new configuration to deter¬ 
mine which currently outstanding transactions contain 
the faulty server as the next hop in the forward direction. 
These transactions are retransmitted to move the transac¬ 
tion toward a commit or abort state. To prevent duplicate 
messages from affecting correctness, servers maintain a 
list of committed and aborted transactions. Upon receipt 
of a retransmitted forward-bound message, a server will 
first consult this list and answer with a commit or abort 
message if appropriate. Otherwise, the server processes 
the message to move the transaction closer to commit¬ 
ting; typically this will entail sending another message 
forward in the chain, or waiting for a previously sent 
message to return “commit” or “abort”. Overall, the co¬ 
ordinator and servers will repeat this process as neces¬ 
sary until transactions eventually commit or abort. 

3 Implementation 

We have fully implemented the system described in this 
paper. The code base consists of 130,000 lines of 
code, approximately 15,000 lines of which are devoted 
to processing transactions. The Warp distribution pro¬ 
vides bindings for C, C-H-, Python, Ruby, Java, Go, and 
Node.JS and supports a rich API that goes well beyond 
the simple get/put interface of typical key-value stores. 
A system of virtual servers maps a small number of 


servers to a larger number of partitions, permitting the 
system to reassign partitions to servers without reparti¬ 
tioning the data. The implementation uses a replicated 
state machine as the coordinator to ensure that there are 
no single points of failure. 

3.1 Rich API 

Acyclic transactions naturally support an expanded API 
that enables complex applications. The expanded API 
includes support for rich data structures, multiple inde¬ 
pendent schemas, and nested transactions. 

3.1.1 Data Structures 

The discussion in Section |2] presented all operations in 
a acyclic transaction as either a read or a write on an 
arbitrary string, but our implementation goes much fur¬ 
ther to support many data structures commonly used in 
modern applications. Warp provides programmers with 
integer, float, list, set, map, and document types as well 
as atomic operations on each of these types that enable 
fast concurrent operation. For example, it is possible to 
atomically add an element to a list, or perform arithmetic 
on an integer type. A write, then, may consist of any 
of these atomic operations and is not limited to simply 
overwriting the previous value. These atomic operations 
are especially useful for cases where acyclic transactions 
enable low abort and retry rates because they allow ap¬ 
plications to further improve concurrency. 

3.1.2 Independent Schemas 

Acyclic transactions generalize well from operations 
across multiple keys to operations across multiple keys 
in different schemas. In our implementation, applica¬ 
tions may create multiple schemas—which resemble ta¬ 
bles from traditional database systems—and store differ¬ 
ent objects in each schema without any collisions in the 
key space. Clients construct the chain for transactions 
that touch multiple schemas by lexicographically order¬ 
ing servers first by schema, then by key. 

3.1.3 Nested Transactions 

The architecture we have presented naturally supports 
nested transactions with only minimal changes to the 
client library. Nested transactions may be implemented 
by allowing transaction contexts to recursively refer to 
each other. Each nested transaction maintains its own 
locally-managed transaction context with a pointer to the 
parent transaction’s context. Reads recursively query the 
parent context until either a cached value is read, or the 
root context issues the query to a storage server. Writes 
are stored in the transaction context to which they are 
issued. At commit time, the client merges a nested trans¬ 
action into its parent context, by merging the read and 
write sets. Nested transactions abort if the values read in 
the child are modified in the parent or vice-versa. The 
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client sends a acyclic transaction to the storage servers 
only when the root transaction commits. 

3.2 Virtual Servers 

Warp uses a system of virtual servers to map multiple 
partitions of the mapping to a single server. Clients con¬ 
struct their acyclic transaction chains by constructing a 
chain through the virtual servers, and then mapping these 
virtual servers to their respective servers. A server that 
maps to multiple virtual servers in a chain will appear 
at multiple places in the chain, where it acts as each of 
its virtual servers independently. Within each physical 
server, state is partitioned by virtual server, so that each 
virtual server functions as if it were independent. Vir¬ 
tual servers enable the system to perform dynamic load 
balancing more efficiently. 

3.3 Coordinator 

A replicated state machine called the coordinator par¬ 
titions the key space across all data servers, ensures 
balanced key distribution, and facilitates membership 
changes as servers leave and join the cluster. Since the 
coordinator is not on the data path, its implementation is 
not critical to the performance of acyclic transactions. 

The coordinator partitions data across servers and en¬ 
sures balanced key distribution by using copyset replica¬ 
tion Eh to group servers into replica sets. Each indepen¬ 
dent schema is partitioned across the generated copysets 
to create an object-to-server mapping. The coordinator 
over-partitions the key space to enable it to remap par¬ 
titions from over-burdened replica sets to under-loaded 
replica sets if necessary. 

As servers join and leave a cluster, the coordinator re¬ 
generates copysets to respond to new members. Servers 
dynamically compute the previous and next servers in 
each acyclic transaction’s chain using the mapping; when 
the mapping changes, servers retransmit transactions 
whose chain changed. Every message carries the con¬ 
figuration’s version to enable clients and servers to de¬ 
tect and re-route out-of-date requests using an up-to-date 
configuration. The mapping is changed incrementally, 
ensuring that each subsequent mapping overlaps with the 
previous mapping, which ensures that some replicas in 
each inlined chain will overlap as well. Thus, servers 
are always able to integrate new nodes without violating 
the assumptions used to construct acyclic transactions’ 
chains. 

The coordinator is implemented on top of the Repli¬ 
cant replicated state machine system. Replicant uses 
chain replication to sequence the input to the state 
machine and a quorum-based protocol to reconfigure 
chains on failure. The details of Replicant are beyond 
the scope of this paper; the function of the coordina¬ 
tor could also be built on configuration services such as 
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Figure 4: Total transactional throughput of the three systems. 
Warp outperforms MinlDex by a factor of 4, and achieves 75% 
the throughput of HyperDex, which Warp uses as its underly¬ 
ing key-value store. Warp averages approximately 7,500 trans¬ 
actions, or more than 225,000 individual key operations, per 
second in this benchmark. 


Chubby |3], ZooKeeper 12^ . and OpenReplica ||3l. 

4 Evaluation 

In this section, we evaluate Warp’s performance and seal- 
ability using both macro and micro benchmarks. The pri¬ 
mary focus of our evaluation is on examining the per¬ 
formance of Warp transactions relative to other trans¬ 
action processing techniques. To that end, we imple¬ 
mented Sinfonia’s mini-transactions ^ on top of Hyper¬ 
Dex, hereafter referred to as MiniDex. Because Warp 
builds upon HyperDex, and because native HyperDex 
outperforms many NoSQL databases, we ensure a true 
apples-to-apples comparison by building all systems us¬ 
ing the same code base. We also compare Warp to Hyper¬ 
Dex, even though the latter offers no transactional guar¬ 
antees. The client-facing interfaces and the benchmark 
code is identical for all three systems. 

We performed our experiments on our dedicated lab- 
size cluster consisting of thirteen servers, each of which 
is equipped with two Intel Xeon 2.5 GHz E5420 pro¬ 
cessors, 16 GB of RAM, 500 GB SATA 3.0 Gbit/s hard 
disks, and Gigabit Ethernet. The servers are running 64- 
bit Ubuntu 14.04. Each storage system was configured 
with appropriate settings for a real deployment of this 
size. This includes setting the replication factor to be the 
minimum value necessary to tolerate one failure of any 
process or machine. Both the coordinators and the stor¬ 
age servers can each tolerate one failure. All systems 
provide strong consistency guarantees, which MiniDex 
and Warp extend across multiple objects. 

4.1 TPC-C Macro Benchmark 

The industry-standard TPC-C benchmark simulates an 
e-commerce application by specifying a mixed transac- 
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Figure 5: Write operations have similar latency in HyperDex and Warp because the cumulative lengths across all chains formed 
within a transaction are the same. A HyperDex transaction with O operations makes O chains of length /+ 1. A Warp transaction 
will make a single chain of length O x(/+l). 


Profile 

R 

W 

RMW 

% Mix 

New Order 

12 

3 

11 (1) 

45 

Payment 

0 

1 

3(2) 

45 

Order Status 

12 

0 

0 

5 

Stock Level 

201 (1) 

0 

0 

5 


Table 1; A summary TPC-C workload. For each transac¬ 
tion profile, the chart shows the average number of read-only 
(R), write-only (W), and read-modify-write (RMW) opera¬ 
tions. The TPC-C workload will randomly perform transac¬ 
tions accordingo to this distribution. The values in parenthesis 
specify the number) of district or warehouse objects per trans¬ 
action. 


tion workload. The workload specified by TPC-C is in¬ 
herently difficult to process with optimistic concurrency 
control, because it includes both read-heavy and update- 
heavy transaction profiles and the update-heavy trans¬ 
actions intentionally contend on a small number of hot 
keys. For instance, the new-order transaction gener¬ 
ates the order’s identifier using a sequentially-increasing 
counter associated with one of one-hundred districts. 
The payment transaction increments the year-to-date to¬ 
tals for one of one-hundred districts and one of ten ware¬ 
houses. The contention and interaction between the new- 
order and payment transaction profiles is what makes the 
TPC-C benchmark a compelling choice for testing new 
optimistic protocols. 

At the core of the benchmark are multiple transaction 
profiles which each represent a different type of applica¬ 
tion logic. Table[T]provides an overview of each transac¬ 
tion type. The values in parenthesis specify the number 
of district or warehouse objects per transaction. The bulk 
of the workload stems from the new-order and payment 
transaction profiles. These profiles simulate a customer 


placing purchase orders, and subsequently paying the in¬ 
voice. Our implementation of TPC-C retains as much 
functionality of the benchmark as is reasonable to im¬ 
plement on a key-value store. In total, the implementa¬ 
tion consists of approximately 1100 lines of Python code 
that execute client side using the Python bindings to Hy¬ 
perDex, MiniDex, and Warp. We omitted the “delivery 
transaction” profile because the TPC-C benchmark spec¬ 
ifies that it be performed by a background process that 
would be handled by a messaging queue in a real de¬ 
ployment. Because we chose to retain most of the TPC- 
C benchmark’s behavior, our results are incomparable 
to others in the literature that simply perform new-order 
transactions i,0. 

We deployed the TPC-C benchmark with its default 
setting that includes 10 warehouses, which are very con¬ 
tended keys, and 100 districts, which are somewhat con¬ 
tended keys. Each new-order or payment transaction 
includes one warehouse and one district in the set of 
keys that it reads, modifies, and writes. For the trans¬ 
actional systems, these keys will be the ones most likely 
to introduce transaction abort and retries. For the Hy¬ 
perDex workload, there cannot possibly be any conflict 
because the reads and writes may proceed in any order 
without transactional consistency. Intuitively, we expect 
that the performance of HyperDex provides an upper 
bound on throughput and a lower bound on latency for 
all experiments, because MiniDex and Warp add strictly 
more mechanism on top of the existing code. Conse¬ 
quently, the HyperDex upper bound allows us to objec¬ 
tively gauge how much overhead each system adds to the 
baseline. 

Figure |4] shows the overall transactional throughput 
for HyperDex, MiniDex, and Warp. The experiment 
shows that that Warp achieves a throughput that is four 
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times higher than MiniDex, and close to 7,500 transac¬ 
tions, or 225,000 operations, per second. To put the fac¬ 
tor of 4 in perspective. Warp achieves 75% the through¬ 
put of the non-transactional system on which it builds, 
while MiniDex does not even realize 20% of its poten¬ 
tial. 

The intuition for why Warp is so much more efficient 
is two-fold: hrst. Warp’s transaction management allows 
more concurrency than is possible with MiniDex; and 
second. Warp’s communication costs are similar to those 
of the baseline and require no additional messages. Both 
systems construct chains to write data into the system, 
where each link in the chain equates to a network round 
trip. Where HyperDex will construct one chain of length 
/ -f 1 for each of the O operations. Warp will commit the 
operations through a single chain of length (9 x (/ + 1) 
to commit the transaction. Thus, in the common case 
of no aborts and no retries. Warp requires no additional 
round trips beyond those required for a write within Hy¬ 
perDex. Figures |5a] and |5b] show latency CDFs for the 
new-order and payment transaction prohles. We can see 
that for both transaction types, the latency of HyperDex 
and Warp follow a similar trend, while the latency of 
MiniDex is approximately hve times higher. 

Because transactions must validate read operations, 
there’s an additional cost to performing a transactional 
read that is not paid for non-transactional workloads. The 
read that the Warp client library performs to pull the ob¬ 
ject into the transaction context is the same cost as the 
read that the non-transactional code will perform. The 
Warp client then validates the read at commit time. In 
figure |6l we directly quantify the latency profile of the 
read-only “order status” transaction. We can see that 
Warp’s latency is approximately three times higher than 
the non-transactional measurement, while the MiniDex 
latency is approximately six times higher. 

Overall, the reason MiniDex achieves lower through¬ 
put and higher latency is because mini-transactions are 
more likely to abort. We observed, on average, only 
5% of transactions complete without aborting or retry¬ 
ing at least once, and we’ve included the time taken to 
retry transactions in the above numbers for all systems. 
Because all three systems use the same benchmark and 
baseline code, the performance difference is solely the 
commit protocol in use. MiniDex cannot permit mul¬ 
tiple transactions to prepare for the same key simulta¬ 
neously, forcing transactions to abort or wait, which in¬ 
creases the latency by a small constant multiplier. Warp 
permits these transactions to prepare simultaneously, en¬ 
abling it to complete all transactions without aborting. 

Although it may seem possible to relax the mini¬ 
transactions protocol to permit transactions to prepare 
for the same key simultaneously, doing so would break 
serializability. A modified MiniDex would require ad- 



Figure 6: Read operations have different latencies inside and 
outside of a transaction. A non-transactional read may be di¬ 
rectly performed against the server. A transactional read in¬ 
cludes that latency, plus the cost of validating the read at com¬ 
mit time. 


ditional mechanisms to prevent the potential cycle illus¬ 
trated in Figure 13 as concurrently prepared transactions 
could commit in different orders on different servers. 
Even HyperDex’s atomic operations cannot enable such 
a relaxed commit protocol because they cannot affect the 
order in which operations occur on different servers. 

4.2 Micro-Benchmarks 

In order to gain insight into the behavior of Warp’s 
acyclic transactions, we examine the results from sev¬ 
eral targeted micro-benchmarks. In all of these micro¬ 
benchmarks, objects have 12 B keys and 64 B values, 
and are constructed uniformly at random. Ten million 
objects are preloaded onto the cluster before performing 
each benchmark. 

4.2.1 ReadAArite Ratio 

In order to quantify the effects of the read/write ratio 
on a transactions’ throughput, we constructed a micro¬ 
benchmark that varies the read-write ratio for operations 
of constant size. This micro-benchmark constructs trans¬ 
actions that involve exactly eight objects, and randomly 
read from or write to random objects. Each operation 
is randomly chosen to be a read or a write so that the 
total percentage of write operations matches the inde¬ 
pendent variable. In HyperDex, a read incurs one round 
trip, while a write incurs / -I- 1 round trips. Thus, we 
expect that a write-heavy workload in HyperDex will 
achieve lower throughput than a read-heavy workload, 
with all other factors hxed. Because of the validation 
step, we expect Warp transactions to be largely a mat¬ 
ter of the latency of the commit. In Eigure [T] we see 
the average throughput for Warp is 150,000 transactions 
per second, regardless of the workload, while HyperDex 
performance increases as read percentage increases. It 
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Percent PUT operations 


Figure 7: The ratio of read/write operations does not materi¬ 
ally affect the throughput for transactions. 



Figure 8: The total throughput of Warp is dependent on the 
throughput of the underlying key-value store, and of the trans¬ 
action size. This graph shows the throughput of a 100% write 
workload as the number of keys in a transaction increases. 


demonstrates that the performance is a function of the 
transaction protocol and not the read-write ratio. 

4.2.2 Transaction Size 

Naturally, the use of chains introduces a tradeoff; as 
transactions grow to contain more keys, the length of 
the resulting chains naturally increases as well. Figure [8] 
quantifies this tradeoff by constructing write transactions 
with different numbers of keys. We employ the same 
micro-benchmark from the previous section, and use a 
100 % write workload. 

To test the performance impact of transaction size, we 
modified our previous microbenchmark to vary the num¬ 
ber of keys in a transaction rather than the read/write ra¬ 
tio. In this experiment, the microbenchmark issues trans¬ 
actions with a configurable number of put operations on 
random keys. Figure[8]shows that, as expected, the num¬ 
ber of operations per second is mostly independent of 
the transaction size. This demonstrates that longer trans¬ 



Figure 9; Warp is a scalable system. This graph shows the 
aggregate throughput of the system as servers are added. With 
each additional server, the overall throughput increases propor¬ 
tionally, exhibiting linear scaling. 


action chains do not introduce additional overhead, and 
that, for this workload, the transaction rate is a linear 
function of the transaction size. 

4.2.3 Scalability 

The performance of acyclic transactions should scale lin¬ 
early with the number of servers in the cluster, as the 
number of servers that participate in a acyclic transac¬ 
tion is dependent only on the transaction size. Adding 
more servers to the cluster should therefore yield a pro¬ 
portional increase in performance by spreading the work 
across more servers. Figure |9] shows the aggregate 
throughput of a two-key transaction from our micro¬ 
benchmark with different cluster sizes. Not surprisingly. 
Warp throughput scales linearly with cluster size. 

4.3 Summary 

Overall, the acyclic transactions protocol provides a low 
overhead means of ensuring serializable transactions in 
a distributed key value store. Despite providing ACID 
guarantees. Warp achieves throughputs close to those as¬ 
sociated with non-transactional data stores. This is pri¬ 
marily due to the way acyclic transactions constrict data 
to chains that avoid bottlenecks at dedicated transaction 
managers. This level of performance does not require 
static analysis of applications, or dependency on syn- 
chronicity assumptions, or reliance on loosening appli¬ 
cation semantics. 

5 Related Work 

Transaction management has been an active research 
topic since the early days of distributed database sys¬ 
tems. Existing approaches can be broadly classified into 
the following categories based upon the mechanisms em¬ 
ployed and resulting guarantees. 
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Optimistic Concurrency Control: Acyclic transac¬ 
tions are a form of optimistic concurrency control 
because a validation step is necessary to prevent conflicts 
among concurrent transactions. Traditionally, optimistic 
concurrency control schemes have been divided into 
backward-oriented and forward-oriented schemes toll . 
The former checks optimistic reads against previously 
performed writes, while the latter checks optimistic 
writes against concurrently executing, unvalidated reads. 
Warp’s concurrency control is most similar to backward 
oriented OCC, but includes additional validation steps 
beyond those typically employed in backward oriented 
systems because it permits transactions to execute con¬ 
currently. 


Centralized: Early RDBMS systems relied on physi¬ 
cally centralized transaction managers |@]. While cen¬ 
tralization greatly simplifies the implementation of a 
transaction manager, it poses a performance and scala¬ 
bility bottleneck and is a single point of failure. Warp is 
based on a distributed architecture. 


Distributed: The traditional approach to distributing 
transaction management is to provide a set of special¬ 
ized transaction managers that serve as intermediaries 
between clients and back-end data servers. These trans¬ 
action managers perform lock or timestamp manage¬ 
ment ||3l, and employ a protocol, such as two phase- 
commit, for coordination. In the class of two-phase com¬ 
mit protocols. Linear 2PC El is similar to Warp in 
that the communication pattern is arranged along a chain 
of servers. The key improvement of acyclic transac¬ 
tions over existing multi-phase protocols, including Lin¬ 
ear 2PC, is that it does not employ specialized transaction 
managers and permits multiple transactions to execute in 
parallel directly on a subset of storage servers, even when 
operating on the same key(s). 

A recent proposal S suggests physically separating 
the transaction processing component from the storage 
component so that transaction processing remains agnos¬ 
tic to the structure of the storage. The resulting system, 
Deuteronomy llil]] performs this separation so that trans¬ 
action management remains isolated from scaling deci¬ 
sions made at the storage layer. ElasTraS El uses two 
layers of transaction management to separately process 
read-only and read-write transactions, where each layer 
and the underlying storage are independently scalable. 
Separating transaction management from data storage 
does not fundamentally make the transaction manage¬ 
ment more scalable because it does not alter the spurious 
coordination naturally present in the transaction man¬ 
ager. Warp reduces spurious coordination and central¬ 
ized bottlenecks by employing a completely distributed 
protocol. 

Recent work has proposed database systems with 


minimal coordination through the use of a technique 
called X-confluence analysis H. X-confluence requires 
of the programmer a set of invariants that are processed 
by offline analysis to determine a minimal amount of co¬ 
ordination to uphold the invariants. Warp transactions 
are fully serializable, require no programmer-provided 
specifications, and naturally avoid spurious coordination. 


Consensus-based: Recent work has examined how to 
use a general consensus protocol, such as Paxos lEsIl or 
Zab 1241], to serialize transactions in a fault-tolerant man¬ 
ner. Although consensus seems unrelated to transaction 
management, the classic two-phase commit algorithm is 
actually a special / = 0 case of Paxos that cannot tolerate 
coordinator failure lE^I . 

Straightforward application of consensus protocols, 
however, would introduce spurious coordination by ap¬ 
plying a total order across all transactions. Consequently, 
consensus-based systems typicall y u se some combina¬ 
tion of data partitioning Q 1^ |4^. Generalized 
Paxos 1251 or transaction batching j^, ^ to increase 
opportunities for parallel execution. 

Warp uses consensus only to maintain system meta¬ 
state. The acyclic transactions protocol, inspired 
by chain-replication lEl] and value-dependent chain¬ 
ing E3], relies upon consensus for system membership 
and coordination, but not for actual transaction process¬ 
ing. Because consensus is not on the critical path for 
any acyclic transaction, the protocol is able to completely 
eliminate any consensus-induced overhead. 


Synchronized clocks: Some notable systems in this 
space take advantage of synchronized clocks to order 
transactions. Adya et. al. yj] support serializable trans¬ 
actions and use loosely synchronized clocks as a per¬ 
formance optimization. Spanner El uses tightly syn¬ 
chronized clocks, with bounded error, to achieve high- 
throughput and external consistency for transactions 
across multiple data centers. Granola Oldll orders inde¬ 
pendent transactions with no locking overhead or abort 
mechanism, and orders these transactions using time 
synchronization as an optimization. 

Warp is an asynchronous protocol that makes no as¬ 
sumptions about clock synchrony. It is more robust than 
systems which make synchronicity assumptions, and re¬ 
quires less maintenance and operations infrastructure. 
Most systems in this category remain correct should 
synchronicity assumptions be violated, but suffer vary¬ 
ing degrees of performance degradation. A notable ex¬ 
ception is Spanner, which preserves serializability only 
when its assumptions are upheld. 


CUent-managed transactions: Some systems build 
on existing storage by implementing transactions directly 
in the client library. Such systems mediate concurrent 
transactions by embedding additional attributes into the 
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stored objects to enable concurrency control. CrSO 1118 1 
uses HBase versions and a centralized status oracle to 
check for read-write or write-write conflicts at commit 
time. Percolator ll^ maintains Google’s search index 
by storing both data and locks in BigTable. 

The downside to client-managed approaches is that 
they require mechanisms to cope with client failure. 
CrSO requires a background process to cleanup stale ver¬ 
sions of objects written by failed transactions. Percola¬ 
tor uses a background mechanism to break locks held by 
failed processes. Warp incurs no such cost because failed 
clients leave behind no state to clean up. 


Geo-Replication: For geo-replicated storage, many 
systems avoid synchronous WAN latencies by making 
guarantees weaker than serializability. COPS-GT 132 1 
and Eiger 11331] provide read and write transactions, re¬ 
spectively, that commit locally and propagate to remote 
data centers in a causally-consistent fashion. Walter 
implements parallel snapshot isolation using counting 
sets to resolve conflicting versions, similar to commuta¬ 
tive data types S. Warp provides a strictly stronger 
guarantee of general purpose serializable transactions, 
but lacks optimizations for geo-replication. 


Offline Checking: Lynx if^ uses chains for replica¬ 
tion and guarantees serializability, but requires a priori 
knowledge of transactions and static analysis to prevent 
non-serializable executions. The key insight in Lynx is 
that this static analysis can break one application-level 
transaction into many smaller transactions that execute 
piecewise across servers. Rococo S also requires of¬ 
fline analysis of transactions in order to decompose them 
into smaller atomic units, which it then executes in par¬ 
allel. Warp is fundamentally different from transaction 
management schemes that rely upon offline checking be¬ 
cause it guarantees serializability without requiring that 
transactions are known a priori, and without requiring 
any static analysis across all transactions. 


Workload Partitioning: Some systems improve per¬ 
formance by constraining transactions to operate within 
single partitions of the data store. G-Store pro¬ 
vides serializable transactions on top of HBase by group¬ 
ing keys’ primary replicas on a single server so that 
transactions require no cross-server communication. H- 
Store ll^ targets OLTP applications and efficiently sup¬ 
ports such constrained tree applications by guaranteeing 
that transactions are executed by a single server. Warp 
imposes no constraint on transactions, enabling maximal 
flexibility in data placement. 

Mini-Transactions: Sinfonia ^ introduces the mini¬ 
transaction primitive which allows an application to 
specify sets of checks, reads, and writes and commit the 
result using a modified two-phase commit. The pay- 
load of a mini-transaction is the same as the payload 


of a acyclic transaction. Indeed, the protocols can be 
quite similar in their behavior; both are optimistic, both 
require two (or more) phases to commit a transaction 
across multiple servers, and both have the client library 
optimistically execute reads and writes to be validated at 
commit time. 

Where mini-transactions and acyclic transactions dif¬ 
fer is in their commit behavior. Acyclic transactions al¬ 
low multiple transactions that read or write the same key 
to simultaneously execute and commit in a serializable 
order. Additionally, mini-transactions are vulnerable to 
client failure while acyclic transactions are naturally fault 
tolerant. These differences cannot be overcome by sim¬ 
ply loosening the commit requirements within Sinfonia, 
because the resulting mechanism not be serializable. 


Key-Value Stores: Key-value systems are defined by 
their distributed architecture that offers performance and 
scalability, often obtained by avoiding strong consis¬ 
tency or transactional guarantees. This trade-off is of¬ 
ten an engineering decision to mask latency, and is not 
fundamental. Amazon’s Dynamo and its deriva¬ 
tives 127, 3^ guarantee only eventual consistency 
in order to increase write availability by writing data to 
sloppy quorums. Yahool’s PNUTS 11211 makes a slightly 
stronger guarantee of per-object timeline consistency, but 
makes no guarantees across multiple objects. Google’s 


BigTable lllOll provides linearizable access to individ¬ 
ual rows, but does not make cross-object guarantees. 
BigTable’s consistency is the same as HyperDex IB, 
the system Warp builds upon. Warp’s guarantee is strictly 
stronger as it extends serializability across multiple ob¬ 
jects. 

More generally, these NoSOL systems have roots in 
Distributed Data Structures 12311 and distributed hash ta¬ 
bles 1 4^ 4^, 13, 53], which provide efficient access to 
individual objects, usually in the form of a key-value 
store. Other notable work on key-value stores includes 
FAWN-KV 10], a linearizable key-value store built to re¬ 
duce power consumption in storage systems; Comet 
a key-value store that stores clients’ code alongside the 
stored objects; RAMCloud OTII . which builds a key- 
value store for low-latency networks; and FaRM, which 
uses RDMA to build a fast in-memory key-value store 
with single-machine transactions. The goals of these sys¬ 
tems are orthogonal to those in Warp, and the techniques 
could be combined to make a transactional key-value 
store with low power consumption (maximizing trans¬ 
actions per watt), or low latency (minimizing transaction 
completion time). 


6 Conclusion 


This paper describes Warp, a key-value store that pro¬ 
vides one-copy-serializable ACID transactions. The 
main insight behind Warp is a protocol called acyclic 
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transactions which enables the system to completely dis¬ 
tribute the task of ordering transactions. Consequently, 
transactions on separate servers will not require expen¬ 
sive coordination and the number of servers that process 
a transaction is independent of the number of servers in 
the system. The system achieves high performance on 
a variety of standard benchmarks, performing nearly as 
well as the non-transactional key-value store that Warp 
builds upon. 
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